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A statistical tnocel designed to assist elementary 
scnocl principals in tne process ct selection educational areas which 
should receive additional emphasis is presented* Jor each educational 
area, trie model produces an index number which represents the 
expected "value" per dollar spent on an instructional program 
appropriate ter strengthening that area* Although the model is 
explained in terms ot this specific application, the approach 
proposed could also he used to structure similar decision problems at 
the district cr state levels, or in secondary and pre-school 
educational systems. Ine calculation ot the index number tor a 
particular area depends on: (1) the relative importance ot that area 

in terms ot the overall educational goals; (2) the "utility 11 or 
"value" to the decision Raker ot making an improvement m that area, 
given the current level ct performance; (3) the probability 
distribution ot the results ot implementing a particular type ot 
improvement program tor that area, given the current level of 
performance; and (4) the cost ot the program. These factors are 
combined into a statistical formula yielding the desired index 
number. n tie use of these indices as decision instruments is also 
discussed* Statistical data and a bibliography are included* 

( Auth c r/AE) 
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This paper presents a model designed to assist elementary 
school principals in the process uf selecting educational areas 
which should receive additional emphasis. lor each educational 
area, the model produces an index number which represents the 
expected "value" per dollar spent on an instructional progiam 
appropriate Tor st rengtheni ng that aiea. Although tiie model 
will be explained in terms of this spec i lie application, the 
approach proposed in this paper could also be used to stricter*, 
similar decision problems at the district o! state levels, oi 
in secondary and pre- school educational systems. 

The calculation of the index number for a particular area 
depends or, the id lowing 1 actors. [3J the relative importance 
of tli at area; (2) the "utility," or "\alue" to the decision 
maker, of making an improvement in that aiea, given the euireni 
level of performance; (3J the pjohability distribution of the 
results of im; 'o' eating a part’ ular type ol improvement prog] ; 
for that area, given the vuiiont level ot neitormance; and i • 
the cost of the above mentioned pro., rail. Jhe lirst two faciois 
will be discussed i ji tlie t i » >t section ot tins paper. In the 
second section the last two v. \ i 1 be cons: deied, and all loui 
will be combined in a fuiviiJa welding tlie des i : "d inde.\ n; d l . 
Ihe use of these iinliws as an aid to decision raking will then 
be explained. 
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THE ESTIMATION OF UTILITY 



This section describes how the ''utility"* (of the decision 
maker) for the current state of the system is estimated. In 
our particular application, the system is an elementary school, 
the decision maker is its principal, and the state of the system 
is represented by the level of educational achievement of the 
school. As we shall see, the process of estimating utility pre- 
supposes the existence of (1) a well defined hierarchy of system 
objectives, and (2) adequate devices for measuring the degree of 
achievement of these objectives. We are then left with the pro- 
blem of transforming performance measurements into a single 
(utility) number; this number reflects the decision maker’s 
"satisfaction" with the state of the system as represented by 
these performance measures. 

The formal structuring of a decision problem (in our case, 
the selection of educational areas to be emphasized or strength- 
ened) must begin with the statement of an overall objective -- 
perhaps one as vague as "promote the good life". By asking how 
a given system contributes to the achievement of this "meta- 
objective", a hierarchy of primary objectives, secondary objec- 
tives, goals, and subgoals can be identified. For example, in 
elementary education, the primary objectives might be (1) the 
development of the child's personal or affective qualities, (2) 
the teaching of the basic skills required for further educational 
success, and (3) the introduction of the child to cultural norms 

* 

For a discussion of the meaning of "utility", see the clasic 
work by J. Von Neumann and 0. Morgenstern (1953) or that of 
Schlaifer (1959) . 
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and values. Each of these three primary objectives can be 
subdivided into the secondary objectives which they logically 
and operationally imply. For example, objective (2) above may 
be subdivided into the teaching of verbal skills and the teach- 
ing of analytical skills. In a similar manner, secondary ob- 
jectives may be subdivided into goals, and the latter into sub- 
goals, etc. This subdivision process should continue until each 
"terminal" (lowest level} objective/goal in the hierarchy can be 
associated with an appropriate device for measuring the system's 
performance in that area. Figure 1 illustrates an example of 
the results of applying this objective subdivision process to 
elementary education. The numbers in parentheses refer to the 
terminal goal areas (in this case, subgoals) listed in Appendix 
I. For a further discussion of this process, as well as of some 
important caveats, see Pardee (1969), especially page 405. 

The performance of an educational system is not usually de- 
termined by a direct measurement of how well it achieves its pri 
mary or secondary objectives. These objectives are too broad in 
nature for the development of valid and reliable measuring in- 
struments. Instead, the state of the system may be assessed in - 
directly by measuring its performance m each terminal goal area 
for which adequate measuring devices exist. The Center for the 
Study of Evaluation at UCLA has investigated the existence of 
test instruments for each of the attributes listed in Appendix I 
(see Elementary School Evaluation Project, Booklet III). The re 
suits of this study indicate that numerous instruments are avail 
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Personality Development (1A-43) 
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Figure 1. illustrative Hierarchy 
of Objectives 



able in the skill and cognitive areas, with their number and 
validity decreasing in the affective areas. However, continuing 
improvement in the number and validity of tests in all areas can 
be expected. 

The results achieved on the various test instruments are 
generally expressed in terms of percentile scores or rankings on 
a national basis. Consequently, there is ro absolute, invariant 
scale against which to measure performance, and we must take these 
scores to be our "raw" system performance measurements. It seems 
reasonable to assume that the "worth” or "value” (to the princip, 1 
of a given percentiie score depends strongly on both (1) the par- 
ticular goal area involved and (2) his aspiration level for that 
area. Since past achievements of a school depend to some extent 
on such exogenous input factors as the socioeconomic status of 
the parents, the location of the school (urban vs. rural), the 
region of the country, etc., one can reasonably expect these 
factors to influence the principal's aspiration level (and lienee 
his utility for performance measurements) in each goal area.* 

So that the model will be sensitive to these environmental i ac - 
tors, performance data is currently being collected on schools’ 
categorized according to environmental characteristics (see 
hlenen ta vy .School Evaluation Project, booklet IV) . 

Hi is will be discussed in a forthcoming Technical Report on the 
possible impact of such factors on the shape of the utility func- 
tion for a g i yejy g oal a re a , 



. c . 



If v/e accept the (percentile) scores obtained on standard- 
ized tests as adequate measures of the school's performance in 
the associated goal areas, it remains to be determined if these 
scores can be used directly in estimating the principal's util- 
ity. The contents of the previous paragraph and the following 
observations suggest that they cannot; that is, the scores must 
be transformed before they can be used for that purpose: 

1. Given results in a particular goal area and ignoring 
all other results , it is clear that a score of 80 is 
considered twice as "good" as a score of 40. Also, 
it may not be true that a score increase from 40 to 
50 has the same "value" as an increase from 80 to 90. 

2. The "worth" of an increase from, say, 40 to 50 per- 
centile points in two different goal areas may not 
be the same, (i.e., the principal may not be indif- 
ferent between these two outcomes) . 

The three words, "worth", "good", and "value" are often 
used synonymously with the term utility. However, in the last 
two paragraphs, these words have been used to express the util- 
ity of the decision maker with respect to o nly one goal area, 
and not the system as a whole. A key assumption of this model 
is that the principal's utility for a set of n percentile scores 
(one for each goal area, thus characterizing the state of the 
educational system) is simply the sum of his utility* for each 

When these utilities are measured on appropriate scales. 
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of t lie individual scores; therefore, it is evident that o;:r next 
task is to transform the area scores into their associated "area 
ut .i 1 i ty values". 



flic above assumption about the decision maker's utility is 
equivalent to saying that his utility function is additively sep- 
arable*. \ f we let 



f ■ i' ’ 1 -= 

L 



w. S tlie 
1 



the number of terminal goal areas; 

t lie (percentile) score obtained on a standardized list 

appropr iat c for measuring performance in area i; 

the principals’ s ta n da rd * *u t i 1 i t / function for area i, 

i.e. , it transforms tlie score for area i (a^) into a 

number between 0 and 1 ( f . (a . ) . ; 

11 n 

"weighing factor" for area i. If we require that l k. 

i = ] 1 

this weight expresses tlie re lativ e importance of 

area i with respect to the whole set of areas. The 

number can also be viewed as the proper "scaling" 

factor for the s tandard utility i unction f ^ (*J such 

that the principal's utility for the score a. As given 

bv tin.- scaled utiiitv function f (a . ) - v. . f . (a . 1 ; and 

f(a^,...,a^} _ the principal's utility for the set of 

n scores 

a : i = 1 , * - - , n 

i 



then Uie additive utility assumption says that 



* 

See Appendix II. 

*fhe utiiitv of a score of "0" is 0; tlie utility of a score of 
"100" is f. 



O 
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(1) f(a l ,a 2 ,---,a n ) - qUp ♦ f 2 (» 2 ) *-- + f n (a n ) 
or equivalently: 

(2) fCa lf a 2 ,---,a n ) = w 1 f 1 (a 1 )*w 2 f 2 (a 2 )+--+w n f n (a n ) 

To transform the area score, a., into its associated area util- 
ity value, f^(a^), we now need to determine the constant w^ and 
the function f • ( * ) . 

The values of the w • ' s may be ot ained by asking the deci- 
sion maker to compare the relative importance of goals on the 
same level within the hierarchy. For example, according to 
figure 1, the importance of personal development would be com- 
pared against the importance of cultural values and basic skills; 
at a lower level in the hierarchy, similarly, the importance of 
mathematics would be compared against that of science. The final 
weight for a sub-goal area would be obtained by multiplying its 
relative weight within its level of the hierarchy by the relative 
weight of its associated goal at the next highest level, times 
the relative weight of the next highest level goal/ob i ect i ve , etc. 
More detailed procedures for obtaining these weights are given in 
Pardee (1969). In addition, the problem of obtaining and synthe- 
sizing different weights (for each goal area of interest) from 
different groups which interact with the system (e.g., parents, 
teachers and administrators) is also considered in this document* 
The function f^(’) can only be approximated through the anal 



For an alternate procedure, see reference, Elementary school 
Evaluation Project, Booklet 111. This procedure is illustrated 
in terms of the 106 subgoals listed in Appendix I . 
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vsis of empirical data,* However, a few statements can be made 
about its expected shape- It seems reasonable that , defined 
in terms of a percentile score, will be considered to be of great - 



Consequent ly , v;e may assume that the function f ^ ('J is mono- 
tunically increasing on the closed interval [o, i»g , where the 
numbeis in that interval refer to percentile scores. Additionally, 
t he i e appear to be two particular percentile scores on a . andard i zed 
test which serve ^s aspiration levels for the school principal: 

(1’) the national norm (50th percentile score ) and (2 J the norms 
for schools of a particular "type,” as characterized by the 
various environmental factors discussed earlier. Interviews with 
principals and the other individuals associated with elementary 
school systems indicated that an increase (in percentile score) 
of a g i * a\ aj^ioun t from a point below the national norm is consid- 
ered to be of s i g n i f i c a n 1 1 y g r e a t e r v a 1 ue than the s ame amount of 
ipcre,is£ from a point above the national norm. This suggests 
that tiie slope of f. ( * ) is steeper at points below the 50th per- 
centile score than at points above this score. A similar behavior 
may be "expected" with respect to the "environmental" norm; how- 
ever, current data limitations prevent us from verifying this 
hypothesis Ibis prediction of a decreasing slope for the utility 
function as the percentile score increases is also consistent with 
the "law of diminishing marginal utility” which has been empirically 

* lo i a discussion of several alternative approaches and a complete 
bibliography, sec Fishbuin (190'’). lor a discussion of how this 
w;is done in chi s particulax project, see a 1 o r the on i ng Technical 
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er value as It increases; that is, li a a^, then iA(:w) fb (a2). 
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rigurc 2 - A Possible Form of f\ (‘) 



O 

ERIC 

12 



- 10 - 



verii'ied in numerous studies in an economic context. One 



possible frrm of f ^ (*) is shown in Fig. 2. 



EXPECTED CHANGE IN UTILITY AS 
A DECISION CRITERION 



This section describes how estimates of the decision maker's 
utility for various performance levels of the system may be com* 
bined with estimates the effects cn system performance of 
implementing various ''programs" to provide a guide for decisions. 
In our particular application, the decision problem is to select 
educational goal areas which should receive more emphasis. The 
key assumption implicit in this discussion is that the decision 
maker prefers actions which maximize his expected utility. 

It is reasonable to assume that a particular goal area will 
be selected for increased emphasis because (1) there exists an 
educational program (e . g. , a new set of work books, the Sullivan 
reading program, a computer assisted arithmetic program, etc.), 
which lias a 'reasonably good chance" of improving the student's 
performance in that area, (2) a "significant" increase in the 
percentile score in that area results in a "significant" increase 
in the decision maker's utility, and [3) the cost cf the program 
is within the school budget. 

Because of the interaction among the exogenous input factors 
it is impossible to state that "the adoption of a program of type 
i in area i will increase the performance from a° to a"? + c ■ . 1 
Instead, the potential results of adopting a program of type i in 
area i should be described by the conditional (or posterior] 
probability distribution of the scores which would be obtained 
upon retesting the students after implementation of the program. 
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bs , . this distribution, one could calculate the probability of 
-■chi 'v:r,g a spccilied result, ’Vo assume that the random variable 
a', [representing the score to be obtained upon retesting) has a 
probability density function which depends on 3 factors: [1} the 

particular subgoal area (i), (2) the current level of achievement 

(a^ c ), and ( 3 } the particular type of program selected (j), and we 
denote this conditional probability distribution by g^ f a ’ . 'i ^ 0 , j ) * 
3 or example, the subgoal area may be "operat ions with integers," 
tiie current level of achievement may be "4 0th percentile ranking," 
and the type of program may be "a computer assisted arithmetic 
program." A possible form of g ^ (a * ^ |40,i) is shown in Figure 3. 




Figure 3 

A Possible Form of a 1‘ostcj ior Probability Distribution of Scores 

Unfortunately, it is extremely difficult to obtain objective 
information which would yield an estimate of this probability dis- 
tribution. However, subjective information may be used to approx- 
imate the lorn of g^ 1 ^ \ [i \* j ) . Persons who have observed the 

impact of new educational programs nay be asked for their estimate 
of the probability of achieving a specific change in performance 
as a result of adopting a program of type i in area i, when the 
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current level of performance is a? * For example, these "experts' 
may estimate the probability of an increase of from three to 
five percentile points in area i, given a? and program j, to be 
.3, This implies that 



(3) 




S’.j (.V i jd;i' 



U’e will denote a subjective estimate of the value of the integra 

of C aT il n °»j) over a particular interval, by I\ ('■'^|aV l j 

By obtaining similar estimates for K non -over lapping intervals 

which cove]' the range of a 1 . , we can obtain a discrete approxima 

tion to the desired distribution. By varying the size and number 

of the intervals we can make the approximation as "close" to \]\r 

continuous function as we desire.** In addition, we natural Jv 
K 

require that E F. f . |a?,j}=l, 
k= 1 Kl * 

Ihe theoretical probability distribution may be used to 
determine the expected change in utility, resulting from the 
i ■ ip 1 enen t at ion of program j in area i given that the current 
level of performance is a . , as follows: 



' P l.UU.) 







f . ( a u J 
] ^ r 



wh ere F c x p e c t c d v a 1 u c op e r a tor, 

‘Ah ^ change in utility in area i, 



that the symbol f., carries i n forma t i on about both tin 
location, and the size of the interval of interest, 

*"Sce Sehlaifer (1FS9) for a further discussion of this topic. 

*"*Sec Appendix I] for a theoretical discussion c f the as sump 
^ lions implicit in this formulation. 

ERIC 



15 



13- 



and tlie other variables are identical to those defined in the 
previous section. If the discrete approximation to 
is used, expression f4) simplifies to 



15) li(;di.) .k. 
where the value of 



i i 

k- 1 



5 i (/ k 5 ' l ’i ( k| a i 



1 i ,a P 



1M ( c J may be approximated by the value o f 
f . ( 0 a t the midpoint of the interval ^ . 

Since it is anticipated that decisions must be made within 
a budget constraint, an additional modification of expressions 
(4) and (5) is required. Assume that the cost of a program of 
type j lias been estimated to be c . Dividing these expressions 
by Cj would provide an estimate of the expected change in util- 
ity per dollar associated with a program of t y p e i i n a r e a i , 
given af. Uiis figure represents a more desirable guide to 
action, e spec i all}* if the pi og rams in question are divisible.* 
To summarize, the implications of expression (5J may be 
clarified if it is written in an equivalent f o i \w and d i v i d e d b y 
c . as s how n in ( 6 } . 



(bJ 



kUU.) w. 

^ i ■ i 



c • 



K 



h- 1 



‘AV 



t . { a 1 

l v l 






i lie numerator is the product ol the neasuie ot i dative mpor- 
tance, w. , t lie change in utility associated with a given change 



by divisible, we mean that they mav be adopted in part. Methods 
for dealing with this problem ii divisibility is not a reason 
able assumption will be considered in a forthcoming report. 




G 
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in performance, f-(tjJ - f-(a^), and the subjective probability 
of that change, P.(A, | a . , j ] . If tlie probabilities that the re- 

suiting score will fall in each of several disjoint intervals 
Aj. are estimated, tlieir product with, their associated measures 
of change in utility must be summed Ln words, expression (6) 
for the expected change in u llity per dollar is approximately 
equivalent to 

~a hie asure oT tTTe ~ cli aiige in u t Hit y 

t lie importance • associated with the 
■ of area i adoption ot’ a program 

o f typ e j in area i__ 



' fife cost of 
a program 
of type \ 

CuM'i.usiux 

“Jhe model which was described in the preceding sections pro- 
vides the decision maker t s »n elementary school principal) with 
index numb tM’S which repie>enl estimates u i the expected changes 
in his utility for the per i ui nance ul his scb.ool v, h i Hi would re- 
sult from t lie aduption ot p.ritiuiai t\pes ul pi og rams in paitiv- 
ular areas. It is felt that ihese index nunbeis would provide 
valuable information in his dealing with tie problem of identi- 
fying areas in which action should be consideied, and identifying 
the types of programs which would piovide t lie greatest expected 



tlfe p r oTialfi lit)* 
of achieving 
that change 
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contvibution to the achievement of his instructional goals. 

The actual use of the model by an elementary shcool 
principal could be encouraged through the provision of a series 
of tables containing the values 
K 



for each area, i, for a series of scores, a i , and for several 
types of programs, j* The piincipal would merely be required 
to determine his own measures of the relative importance of each 
area (the k^ t sJ, and obtain the remainder of the information di- 
rectly from the tables. 



k = i 
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AITLNDIX I 

outline op M5 goals oi- elementary school education 



ASPECT I VE 

1 . TEMPERAMENT: PERSON A). 

A. Shym?ss - Boldness 

B . Neurol i c i sm- Ail jus tnent 

C. Gone ral Ac t i vi ty- Lethargy 

2. TEMPERAMENT : SOCIAL 

A . Hope ml c n c e - I n d e p e nd o n c o 

B. Host i 1 i ty - Lr iemU i ness 

C. Social in at ion-Kehell ions nos s 

3. ATT DULLS 

A. School Orientation 

B. Self Est.ce m 

4. NEEDS AND I N T TRUSTS 

A . X cod A c h i c vc m out 

B. Interest Areas 



ARTS - CRAFTS 

5. VALUING ARTS AND CRAiTS 

A. Appreciation e 1’ Arts and (’rafts 

B. Involvement in Arts and Crafts 



0. PUCPUC INC ARTS AND ITA1TS 

A. Representational Skill in Arts and Crafts 
R. Lxprossive Skill in Arts and Crafts 

7. UNDERSTANDING ARCS AND CRAPIS 
A. Arts and Crafts Comp rehens i on 

L. Developmental Under s t and j ny of Arts and Crafts 



COGNITIVE 

S . REASON I XG 

A , Cl ass i f ic at or y Reasoning 
B . Re 1 at ion a I * Impl i eat i mi;i 1 Be a sen i ny 

C. Systematic Reason iny 

D, Spatial Reasoning 

0. GREAT I V I TV 

A . C rent i vc M ex ib i 1 i t v 

B. Creative Llueney 

10. MLMORY 

A. Span and Serial Memory 

B. Meaningful Memory 

C. Spatial Memory 

I ORCIC.N LANGUAGE 

11. PORE I CN BANG 1 1 AC- ]. SKIDS 

A. Reading Gonpr< hens i on of a Eoieiyn J angunye 
N. 0 j’ a 1 Comprehens \ on o i .a Eovviyn Language 

C. Speaking llueney in a F»roign language 

D. IV r i tiny llueney in a foreign Lan.-iaye 



o 

ERIC 

hfflinaffaHaoaa 



20 



18 - 



12. FOREIGN LANGUAGE ASSIMILATION 

A. Cultural Insight throng!, a Foreign Language 

B. Interest in and Application of a Foreign Language 

LANGUAGE ARTS 

13. LANGUAGE CONSTRUCTION 

A. Spelling 

B. Punct nation 

C . Capitalization 

1). Grammar and Usage 
L. Penmanship 

F . Written Expression 

G . Independent Application of Killing Skills 
M. K J ; F J : K I ; N C : ) ; SK1IJ.S 

A. Use of Lata Sources as Reference Skills 

B. Summarizing Information for Reference 

MAT! 11. MAT ICS 

J 3 . AR1 TUMLTIC CONCLP'IS 

A. Coinprehens i on of Numbers and Sets in Mathematics 

B. Comprehension of Positional Notation in Mathematics 

C . Comprehension of liquations and Inequalities 

D. Comprehension of Number Principles 

10. AKITlIMhTK; OPERATIONS 

A. Operations wit!) Integers 

B. Operations Kith Fractions 

C . (Operations with Decimals and Percents 

17, MATHEMATICAL ADM. I (NATIONS 

A. Mathematical Problem Solving 

B. Independent App 1 i ca t i on of Mathematical Skills 

18. GEOMETRY 

A . Ge one trie F ac i 1 i t y 
L . Geometric Vocabulary 

1 P . MEASUREMENT 

A, .Measurement Finding and Making 

B , S t a t i s t i c s 



20. MUSIC APPR1 CIA1 K»N AND INI] KPS 1 
A . Mus i c \pprvc i .it i on 

B . Music Interest and In joy: lent 

21. MUSIC PLRIOKMANl'J. 

A. Singing 

I*. Musical Misti unen t Flaying 

C. Ikince (Rhythmic Response} 

22. MUSIC UN MI I SI AN DING 

A. Aural Identification of Music 
R . Music Knowledge 
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PilVSICAH 'iJHIC'AT 1 ON - CHAHTil * S •H/J'Y 
23. IIHAHTH AX I) SAJ ? 1.TV 

A. Practicing Health and Safety Principles 

B. Understanding Health and Safety Principles 

C. Sex education 

21. PHYSICAL SKI IS 

A. Muscle Cent vo 1 (luvsical Hducat i on ) 

B. Physical Je vc 1 opr,en t and Kell -Being ("Physical hducation) 

2 5 . SPORTSMANSHIP 

A* Group Activity * Sportsmanship 

B. I jit ere st in and Independent Participation in Sports 
and Gaines 

2t . PI1YSICA] 1:1 Hi CAT 1 ON 

A. Unders t and i jig nf Rules and St ra Logies of Sports and 
Games 

B. Knowledge of Physical Hducat ion Apparatus and Equipment 
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Sc i ence 



E. Operational Definitions in 

F. Experimentation in Science 

G. Formulation of Generalized Conclusions in Science 

36. SCIENTIFIC K N OK LED ( ; ] i 

A. Knowledge of Scientific ! ails and Terminology 

B . The Nature and Pi. l pose of Science 

37. SC IEN 1 I F 1 C APPROACH 

A. Science Interest and Appiec.ation 

B . Application of Scientific Methods to Everyday Life 

SOCIAL SI UJ)1ES 

S. HISTORY AND CIVICS 

A. Knowledge of History 

B . Knowledge of Governments 
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APPFNiUX II 

let the state ef the educational system be partially char- 
acterized by the n- tuple (a^ , >a ) of percentile scores obtained 

on tests covering the n subgoal areas.* It is reasonable (and 
indeed necessary) to assume the existence of a cardinal utility 
function f: R llv k\ which assigns to each possible state of the 
system a real number representing its , Vorth n to the decision 

o 

maker. Let the initial state of the system be a° = (:i^°, , a^), 

with corresponding utility number f(a°), and suppose that an im- 
provement p log ran** is administered in each sub goal area, and then 
the students are retested. One would expect the resulting new set 
el scores a' = (a, 1 . — ,n ’) to .ominate*** the old one and the nev; 
utility number f(a T ) to be greater tlian f(a°j. Whether or not 
this occurs depends on various uncontrollable factors, so that 
until the new scores are known, they and their associated 
utility mi ibe r must be treated as random variables. A problem 
cf i)i to re st is there lore to estimate, in ad v ance , the change in 
tli-- decision maker’s utility if, given an initial stale a° , 
improvement programs were administered and tin students re- 
tested a.- discussed above. More inpoj tanlly, we wish to esti- 
m; i.c the "c oil t r il-u t i ci." of each progin.i (to the estimated change 

*t'lrnr 1 \ oilier system attributes should be included for a ’’com- 
plete” ; i on . 

**;\ny program applicable to a specified area will do. It could 
be tiie "best" program available for that area. For purposes 
of this discussion it natters only that the program be speeil'c 
enough to have a cost. 

*** a i ’ l i{ ) ° for al 1 i"l , “ - - , 




r. and a j ” > a j 



for at least one i. 



in the decision Maker's utility) and relate this contribution 
to the cost of that program in a way which will be meaningful 
to the decision maker in hi s attempt to maximize la i s expected 
utility by selecting costly improvement programs, while staying 
within his limited budget. 

It is clear *hnt the "utility contribution” of each pro- 
gram depends on various factors, tlnee jf which are its quality 
relative to that of the programs covering the other subgoal 
areas, the shape of the decision linker’s utility function, and 
t li c initial state of the system. It is therefore no trivial 
matter to estimate these "utility contributions," However, if 
the utility function is additively separable* and if the im- 
provement programs are i independent , ** then 1 he expected change 
in utility is simply the sum of the expected utility contri- 
butions of the individual programs. for the sake of clarity 
and case of presentation we shall show this for n=2, the case 
for larger values of n clearly follows. 

for n=2 then, after the initial testing, we have a 2- tuple 
of percentile scores a c . (a^°, a> c ) and a utility number of 
f (a° ) : if (Uj 0 , a,°). If certain improvement programs are con- 
templated for these two areas, the scores which night be ob- 
tained after retesting ale now i nation variables a'-fa^’, a,') 



b f(aj, a ^ , . . . , a ?i ) = f i (a j ♦ f (a j ♦ . . . ♦ I ^ ( i n 1 ; or equivalently with 
standard rescaling of the indi\idual {H’s. f (a ^ , . . . a ) -w ^ J' ’ j ; 



► w n f ., ( n . . , *■ w f (a ) . 
2 2 nun' 



■No soil lover or interaction cited:-. 
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; t j 



a 2 '/« 1 B ,a 2 -).* 



wit]] joint conditional probability distribution g(a^’, 

Clearly the utility 1'or these new scores f(aj',a 7 ') is also a 
ran dom variable, so that the expected utility of the decision 
maker for the new state of the system a r , given the initial state 
a° and the Improvement programs chosen, is given by;** 

(1) 1; [flap, a 

and the expected change in utility is therefore 




The assumption of program independence means that 
(3 1 gUp , ;i p / a p 'ay^gjUp/apg-pap/a/j 



/a , 



'£) ‘I 



i'(a. 



Jgla, 



/a x 0 ,a/)da/da 



where g^ and g, represent the conditional probability distri- 
bution*** of the random variables (given a^ c J and i 7 ’ (given 

a^°) re spec t i ve 1 v . Combining this assumption with tit at of 
so]) arable utility and substituting in (1] yields 

N> J - j7Tap,a_.'/ap,a,rjj f p a p j + w , f , ( a pTj g p a p / a p ) 

g , (a , ’ / a /',)daj ’da-, r 

l ( a i 1 Jg i (a ] 1 / Ui° )g, (a, ’ / a / )da 1 ’ d 
v; jjp , (a / J g 2 (a 1 f /iij 0 ) g 2 (a , ' / a, c ) du^ f d 



* ill is distribution is unknown to us at this tire. In tins 
Appendix we avoid complicating t lie expressions by not in- 
cluding indices winch would identify the particular programs 

* *As sure continuous functions here. ihc integrations are over 
the interval (0,100). 

* * " 1 lies e distributions are also unknown to us at this tire. 
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' if 1 i( a 1 ' '8i(--i , /ai°)dai , Mv 2 y r f 2 (a 2 r )g 2 (a 2 ' / a 2 ° 
= WjH | t L Ui V a ^)J + 'm’. j~p (a 2 1 /a /7] 

Now substituting in (2) we get 

( 51 1 [ i' 1 1 v.^b |f t fa x ’ / a +v; 2 i: 1 J ^ a 2 ’ ^ a 7 ° l 1 1 t a i ° J + K s f 7 U 7 

. H [i^ f^x , /a 1 °) J • V- a l 0) h +u *2 { H t r 2 (a 2 ' /a 2 ° ^ ] ‘ f 2 Ca 2 0 J 
f 1 ]+w J L[A f,] 

so that the expected ’'total" change in utility is just the sum 
of the expected utility changes for 1 1 1 e sub goal areas. Since 
the as sui, ip 1 1 on of program independence implies that the utility 
change in any subgoal area is solely* attributable to the im- 
provement program selected for that area, we nay take the ex 
, Ovted change in utility foi a given area as a measure of the 
expected value of that program to the decision maker (i.e. , 
its e x p e c t e d utility contribution w - 1 - [ /. f ’ « ] ) , given a certain 
pc 1 fur man co level in tiiat area. 

1 1 1 o e a p e v t e d utility contribution o f a prog r a rn is t h 0 
number we wish to relate to the cost of that program to assist 
the decision maker in the program selection process. In order 
to obtain this number we must estimate h £ h ( a ^ 1 / a i °TJ ** (the 
expected, new, unw eighed utility value m area 1 , given a 1 °) 



*.\ctuull) v. e only mean to say that programs coloring other 
areas do not contribute to the utility change m this area. 
It is perhaps too strong to use the word rr >oIeiv' here as 
there an surely other factors and random disturbances which 
a /feet tins utility change . 




**kecall from the main body of the paper that we have already 

detei mined a. 0 , f and w ; i-1.2. 

3 * 1 1 1 



25 



. pm 1 / °)j directly is to 



for i = 1 , . . , , ;i , sir. c e , as me n t i one d earlier, the g . ( a ^ ' / a ^ ° J r 
are imkmiv.n to us 

One approach in estimating Y. 
specify a certain number of percentile points and then to 
estimate, by questioning the appropriate "experts," the pro- 
babil.it that the students will score n^ + -i. percentile points 
c. r better, al t er being exposed to a particular improvement 
ero ! ia:;i, given an initial score of a- c . Let us denote this 
gienubility by p«. and call it t he probabi lity of success. Then 

a Prob (a r ^a- ° + <5 - | a. ° ] 

‘i v i — i i 1 i. 

and an approx i na l i on to the expected new utility value in area 
i is given by 

(i J ]. If . f a . * / a . 0 l^p - r • (a . c + 6 . J + (1-p.) f 

i l l 1 l i i i v 1 l J i i 

In this approximation, the first tern on the right hard 



side o ! ((>) is a cun so rva t i ve estimate of 



LOO 

j fi c«i ' (*v /:, i 0 ' 

Ai.°* 6 . 



la. 



since the utility function is evaluated at the single point of 
a j c * t j , and we know that this function is none ton i cnl ly in 

Without knowl* dee ui the parameters of c- fa. ’ /a.°J 

'' 1 l i l 



ureas i n;:, . 

it l ] i ■ 1 1 clear idiot her or not ( 1 - p . ) f . ( a . ° ) is a cons e rva t i ve 

* i i i 



l nar.. < 



. f' hg:, 
X 11 



a-, m 






■lev. ever, if thcr 1 



a re 



gorul jea.-'oiis to believe that the variance of gjla-'/a. c J is 
relatively small, then the above approximation to K 



L ' j 
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■ M is coin citable that in the future there will bo sufficient 
historical data to es tin ale this function empirically. Perhaps 
it can bo dr.ne presently if v,t do not condition il on a. 5 . I ov - 
ever, for pu'Scrt purposes we shall restrict ourselves to sub- 
jective estimates of certain parameters of the function, to be 
obtained via questionnaires. 
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scons reasonable. At this point it should be clear how to esti- 
mate the expected utility contribution of a program in a given 



area i. It is given by 

(7) w-ILl.vf.] = w. U;[f. (a.’/a. 0 ; -f. f.-. 

V J 1 1 1 J 1 1 1 V 1 ' 1 i ' 1 * 



w. .p.f. (a.°+;.) + (l-p.)f- fa. 

I *11^1 V K 1 1 J 1 1 



f . (a . c ' 

\ K i 



= w. p. 
j 1 l 



f. (a.'+.-i.) - (a- ) 

j v l i J j v i 




;md all the value s_r o jii i re d for t In s comput a ^icm a re j iow avail- 
able . 

'Hie choice of d is a tactical natter. It could be a number 
such that a- +.i. represents the aspiration level of the decision 
maker for area i, given an initial percentile score of a.'. 

Clearly this aspiration level should be realistic, namely a 
score which lias a good chance of being achieved; (a.'+n) nay, 
for instance, also represent one’s feelings about the mean, 
the nod km or tlie mode of the posterior probability distri- 
bution of scores gj (a . ’/a/')- - i 1 lust; at ing the intimate rc- 
lationsnip between c. and the probability of success 
A3 teniat ivcly , m nay be the smallest number of percentile 
points sucli that a score of a. l + : . would be considered an 
improvement over a. Ja subjective isscssno 5 .* must be made in 
this case as to v. hot her o i' not tike posterior distribution 
g-(a’./aw) has really shifted to the right- -relative to the 
prior distribution g- (a . -when ad takes on tlie value a. 1 '*;.), 

1 i w i ii* 

A third way may be to simply let equal a const an* t , say 5 

percentile points, for all subgeal areas. Clearly there must 



:'. l 3 



27 



be tradeoffs between the amount of work which goes into and 
tile information derived from the implement at ion of each of 
these three possible methods; however, such tradeoffs have not 
yet been thoroughly investigated.* At this time the first and 
third net beds are re conn ended. 



*I : or example it is clear that the first no thud requires less 
work than the second; however, the second nay provide infor- 
mation about t lie instantaneous or marginal rate of increase in 
expected utility associated with a particular program (some- 
what in the fashion of a partial derivative). If this second 
method were applied for several values of 6- , one nay be able 
to obtain information about the "suspected" decreasing returns, 
in expected utility, as the "in tensity” of a program is increased. 
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